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Abstract 

We consider the general properties of the replicator dynamical system 
from the standpoint of its evolution and stability. Vector field analysis 
as well as spectral properties of such system has been studied. Lyaponuv 
function for investigation of system evolution has been proposed. The 
generalization of the replicator dynamics for the case of multi-agent sys- 
tems has been introduced. We propose a new mathematical model to 
describe the multi-agent interaction in complex system, 
keywords: replicator dynamics, Hamiltonian systems, gradient dynam- 
ical systems, multi-agent interaction, Lyapunov functions, complex sys- 
tems. 

1 Introduction 

Replicator equations were introduced by Fisher to capture Darwin's notion of 
the survival of the fittest 1 and replicator dynamics is one of the most important 
dynamic models arising in biology and ecology [21 E| j evolutionary game theory 
|3] and economics traffic simulation systems and distributed computing 7 

etc. It is derived from the strategies that fare better than average and thriving 
is on the cost of others at the expense of others (see e.g. This leads to the 
fundamental problem of how complex multi-agent systems widely met in nature 
can adapt to changes in the environment when there is no centralized control 
in the system. For complex "living" system such problem has been considered 
in [Sj. By term complex multi-agent systems under consideration we mean one 
that comes into being, provides for itself, and develops pursuing its own goals 
8 . In the same time replicator dynamics arises if the agents have to deal with 
conflicting goals and the behavior of such systems is quite different from the 
adaptation problem considered in 8 . In this article we develop a new dynamic 
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model to describe the multi-agent interaction in complex systems of interacting 
agents sharing common but limited resources. 

Based on this model we consider a population composed of n S Z + distinct 
competing "varieties" with associated fitnesses fi(v), i — l,n, where v £ [0,1] 
is the vector of relative frequencies of the varieties (vi, 1)2, ...,v n ). The evolution 
of relative frequencies is described by the following equations: 

dv i /dt = v i (f i (v)-(f(v))), (1) 

where i = 1, n, 

n 

(/(«)> = !>/<(«)■ ( 2 ) 

i=l 

The essence of Q and @ is simple: varieties with an above-average fitness 
will expand, those with a below-average fitness will contract. 

Since vi G [0, 1], i = 1, n have to be nonnegative for all time, the system JJJ, 
J2J is defined on the nonnegative orthand 

«l = {ve K" : Uj > 0} . (3) 

The replicator equation describes the relative share dynamics and thus holds on 
the unit a n-1 dimensional space simplex 



S n = |*;elE^:X>i = lj. (4) 
Let us write the Fisher's model in the following form 

(n n \ 

y^ajjVj - J! a ]kVjV k , (5) 
3 = 1 j,k=\ / 

where ay € K, «i € [0, 1], i,j = l,n, n S Z+. 

One can check the system © can be written in such a matrix commutative 
form: 



Here by definition, 



dP/dt=[[D;P],P\. (6) 



P={(v i v j ) i :i,j = l,nj, (7) 



D = ^diag |^a ife w fc : j = 1, n| 



(8) 
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It can be checked easily that the matrix P GEndP™ is a projector in E n , 
that is P 2 = P for all tGR, what appears to be very important for our further 
studying the structure of vector field © on the corresponding projector matrix 
manifold V |H1E3- I n particular, the expression 10 ensures that there exists the 
time invariant vector subspace ImP € E n in the Euclidian phase vector space 
E n , necessary for the replicating system under regard to be the information 
data conserved. 

2 Vector field analysis 

In order to study the structure of the flow JJJJ on the projector matrix manifold 
P9P let us consider a functional W : V where by definition the usual 
variation 

6V(P) := Sp(D(P)6P), (9) 

with D € EndP", Sp : EndP'^lR 1 being the standard matrix trace. Taking 
into account the natural metrics on V, we consider the projection of the 
usual gradient vector field V^> to the tangent space T(P) under the following 
conditions: 

<p(X;P):=Sp(P 2 ~P,X) = 0, Sp(V^, V^*) \ v = 0, (10) 

holding on V for all X € End E n . The first condition is evidently equivalent to 
P 2 — P = 0, that is P E V . Thereby we can formulate such a lemma. 

Lemma 1. The functional gradient V v ^ r (P), P € P, at condition I|1U|) has 
the following commutator representation: 



V^(P) = [[AP],P]- (11) 

Proof. Consider the projection of the usual gradient V\&(P) to the tangent 
space T(V) of the manifold P having assumed that P € EndP": 

V^(P)=W(P)-V„(A,P), (12) 

where A € EndP" is some unknown matrix. Taking into account the conditions 
|TU|). we find 

V V *(P) = D — A — P(D - A) - (P> - A)P + PD + DP (13) 
= PD + DP + 2PAP. (14) 

where we made use the conditions 

V^*(P) = D - A + PA + AP) 
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and 

P(D - A) + (D - A)P + 2PAP = D - A. 
Now one can see from <|14[) and the second condition in (|12H . that 

PAP = -PDP (15) 
for all P S P, giving rise to the final result 

V V *(P) = PD + DP- 2PDP, (16) 

coinciding exactly with commutator (|11|) . 

It should be noted that the manifold P is also a symplectic Grassmann man- 
ifold fj9"lll0p. whose canonical symplectic structure is given by the expression: 

uj( 2 ) (p) : = Sp(PdP A dPP), (17) 

where dw^ 2 \P) — for all P G P, and the differential form l|17|l is non- 
degenerate [Sm] upon the tangent space T(P). 

Let us assume that £ : P — >R is an arbitrary smooth function on P . Than 
Hamiltonian vector field : V — >T(P) on P generated by this function relative 
to the symplectic structure ()17fl is given as follows: 

X t = [[Dt,P\,P], (18) 

where £ EndP™ is a certain matrix. The vector field X^ : P — >T(P) gener- 
ates on P the flow 



dP/di = A ? (P) (19) 

being defined globally for all t £ R. This flow by construction is evidently 
compatible with the condition P 2 = P. This means in particular that 

-X s + PX^ + X^P = 0. (20) 

Thus, we stated that dynamical system ©being considered on the Grassmann 
manifold P is Hamiltonian what makes it possible to formulate the following 
statement. 

Statement 1. A gradient vector field of the form (|18|1 on Grassmann man- 
ifold P is Hamiltonian with respect to the canonical symplectic structure (|1 T|) 
and certain Hamiltonian function £ : P — >R, satisfying 



V£(P) = [A;,P], D t =D, 

where simultaneously V V ^(P) = X^(P) for all P e P. 
Consider now the (n-l)-dimensional Riemannian space 



APT 1 = Ivi G M + :i = l,n, Jjv* 
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with the metrics 

n 

ds 2 (v) := d 2 ^ \ v (v) = g l j(v)dv i dv j \<p, 

where 

Subject to the metrics on M™ _1 we can calculate the gradient of the 

function 'J : V — >R and set on M™ _1 the gradient vector field 

dv/dt = Vv,*(u), (21) 

n 

where vG-M^ -1 , or = 1 is satisfied. Having calculated i|21[) . we can 

formulate the following statement. 

Statement 2. The gradient vector fields V ' ^ on V and Vg^ on M™ _1 are 
equivalent or in another words vector fields 

dv/dt = V g ^(v) (22) 

and 

dP(v)/dt = [[D(v).P(v)},P(v)} (23) 

generates the same flow on M™ -1 . 

As a result from the Hamiltonian property of the vector field on the 

Grassmann manifold V we get a new statement. 

Statement 3. The gradient vector field VgVE' l|21[l on the metric space M™ -1 
is Hamiltonian subject to the non-degenerate symplectic structure 

J 2 \v):=^(P)\ Mrl (24) 

for all v&AI 2m with the Hamiltonian function £^ : M| m — > R, where £^ := 
£ Im" -1 ' £ : ^ ~~ *^ ^ s * ne Hamiltonian function of the vector field (|18|l on T 5 
. Otherwise if n e Z+ is arbitrary our two flows (|22(l and (12;>ll are on V only 
Poissonian. 



3 Spectral properties 

Consider the eigenvalue problem for a matrix P G V, depending on evolution 
parameter t e M: 

*W = ^, (25) 

where / € M™ is an eigenfunction, A G K is a real eigenvalue P* — P, i.e. matrix 
P G V is symmetric. It is seen from expression Ij22f) that specP(i) = {0, l}for 
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all t € R. Moreover, taking into account the invariance of SpP = 1 we can 
conclude that only one eigenvalue of the matrix P(t), t G M, is equal to 1, all 
others being equal to zero. So, we can formulate the next lemma. 

Lemma 2. The image ImPc E n of the matrix P{t) eP for alH G R is 
k-dimensional fc = rankP, and the kernel kerP C E n is (n — fc)-dimensional, 
where k G Z + is constant, not depending on t G K . As a consequence of the 
lemma we establish that at k = 1 there exists a unique vector /o G E n /(kerP) 
for which 

P/ = /o, / ~E"/(kerP). (26) 

Due to the statement above for projector P : E n ~^E n we can write down the 
following invariant in time expansion in the direct sum of mutually orthogonal 
subspaces: 

E n = kerP® ImP 

Take now /o G E n satisfying the condition (|26() . Than in accordance with 
<|22|) the next lemma holds. 

Lemma 3. The vector /o G E n satisfies the following evolution equation: 

dfo/dt = [£>(«), P(v)] f + C (t)/ , (27) 

where Co : R— >R is a certain function depending on the choice of the vector 
/o G Im P. At some value of the vector /o G Im P we can evidently ensure the 
condition Co = for all t G R". Moreover one easily observes that for the 
matrix P(t) G V one has ^UJ the representation P(t) — /o 8) /o, (/o, /o) = 1 , 
giving rise to the system (JSJ) if Jo := {i^/uj G M+ : j = l,n} G P™. 

4 Lyapunov Function 

Let us consider gradient vector fields on V and V g ^ on M^ 1-1 . It is easy 

to state that the function \& : V -^M. given by ^ and equal on M™ _1 following 
expression 

" n 

i,j=l i=l 
n 

under the condition c G R+ , v i — 1 being at the same time a Lyapunov 

i=l 

function for the vector fields V^,^/ on P and Vg^P on M™ _1 . Indeed: 



G 



.„ T dv, . . 

= (v s *,v s *) T(Mrl) >0, 

rf* dP 

a — = Sp(DdP)/dt = Sp(D^) (30) 

= Sp(D, [[£>, P] , P]) = - Sp([P, £)] , [P, £>]) 
= Sp([P,D],[P,£>]*)>0 

for all t e K , where (., -^(m™- 1 ) is the scalar product on T(M r g 1 ^ 1 ) obtained 
via the reduction of the scalar product (., .) on M"upon T(M™ _1 ) under the 

n 

constraint = 1. 

i=l 

5 Multi-agent replicator system 

In the above we have considered the case when rankP(t) = 1, t e M. It is 
naturally to study now the case when n > rankP(t) = k > 1 being evidently 
constant for all t € M too. This means therefore that there exists some or- 
thonormal vectors f a £ ImP, a = 1, k, that 

k 

P(t) = Y,f*(t)®f a (t) (31) 



for 



for all t e M. Put now / Q := jiyQ^ e E : i = 1, n| e ImP, £ = 1 

all a = l,k. The necessary orthonormality condition < f a ,f/} >= <5 Q/ 3, a,(i — 
l,k, can be automatically satisfied if one put f a ^ := (exp/i), jCl , i = l,n, for a 
general skew-symmetric matrix /i = — /i* in the Euclidian vector space Then, 
evidently, all signs at vector components ±^/vf = f a ,i,i = 1, n will be defined 
exactly for each a — 1, /c. As a simple consequence of the above representation 
for stable vector f a £ E n , a = 1, k one derives that in the multiagent case one 
can not write down the resulting replicator dynamics equations in the terms of 
positive concentration frequencies. The corresponding gradient flow on V then 
takes modified commutator form with the Lyapunov type function variation 
(5* : V ->E as follows: 

k 

S^ = J2Sp(D a SP^), (32) 



a=l 



where by definition D* a = D a E endE n , and P^ = f a (t) <g> f a (t), PWP^ 



fe 



P' Q ^a/3, a,P = l,k, P( a ) = P. The resulting flow on P is given as 



a=l 



dP/dt = V v t>(P), (33) 
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where the gradient V v : V(V) — »T(P) is calculated taking into account the set 
of natural constraints: 

f a jX;P) := ^[^(PaPfl - *a^P/j)] = (34) 

for every a ^ (3 — 1, k. Assume now that k = rankP(t) for all t € K as was 
stated before. Then the gradient flow (|33[) brings about 

k 

dP a /dt=[[D a ,P a ],P a }~ ( p a D a Pp + PpD a P a ). (35) 

0=1, P^a 

The flow (|35|) one can also obtain as a Hamiltonian one with respect to the 
following canonical symplectic structure on V: 

k 

J 2 \P) = Sp{P a dP a A dP a P a ) (36) 

a=l 

k 

with an element P = ® P a G V, natural constraints P a Pp — Pa^ap, a,(3 = 

a—l 

1, k, and some Hamiltonian function H € D(P) which has to be found making 
use of the expression 

-i v ^u (2) = dH. (37) 
Straightforward but tedious calculations of l|37|) give rise to the same expression 

The system of equations (|35|l is a natural generalization of the replicator 
dynamics for description of a multi-agent interaction. They can describe for ex- 
ample economic communities trying to adapt to a changing environment. In this 
approach agents update their behavior in order to get maximum payoff under 
the given matrices of strategies {a^} in response to the information received 
from other agents. The structure of the system is quite different from the 
system of equations The first term on the right hand side describes the 
individual evolution of each economic agent a in accordance to its own indepen- 
dent replicator dynamics and the second term describes average payoffs of all 
others k agents except a. 
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